A Robust Constrained Markov Decision Process Model for Admission Control in a Single Server Queue

This paper presents a robust optimization approach for discounted constrained Markov decision processes with payoff uncertainty. It is assumed that the decision-maker has no distributional information on the unknown payoffs. Two types of uncertainty sets, convex hulls and intervals are considered. Interval uncertainty sets are parametrized allowing a subset of the payoffs to vary within intervals, to reduce the conservativeness of robust policies. We provide formulations to calculate robust optimal policies and present an application to admission control in a discrete single server queueing problem. It is numerically shown that when the queue manager cannot observe the exact rewards from admissions, mixed threshold type of policies are not necessarily robust optimal.